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(54) Apparatus and method for video frame rate conversion 



(57) A method and apparatus for converting 
between different video formats, which provide smooth 
video motion by eliminating unnatural effects which 
could t^e introduced in the process of video format con- 
version. A frame interpolator produces interpolated 
frames from a first video signal given in a first video for- 



mat by using motion vectors of the first video signal. 
From the interpolated frames, a video signal generator 
produces a second video signal in a second video for- 
mat that is incompatible with the first video format. 
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Description 

BACKGROUND OF THE INVENTION 

1 . Fieid of the Invention 

[0001] The present invention relates to an appara- 
tus and method for converting between different video 
formats, and more particularly, to an apparatus and 
method for converting between different video formats 
to enable visual communication between remote users 
of different television systems. 

2. Descripti on of the Related Art 

[0002] There are three major standard television 
systems available today: NTSC, PAL. and SECAM. 
NTSC is used in the United States and Japan, while 
PAL and SECAM mainly in European nations. For com- 
parison, FIG. 18 shows some key specifications of 
those three different video standards, including: frame 
frequency, the number of effective picture elements 
(pels or pixels) per line, and the number of effective lines 
per frame. Although PAL and SECAM seem to share 
some common parameters, SECAM is actually incom- 
patible in operation with PAL. 

[0003] As noted above, different video formats are 
used in different groups of nations, and to solve this 
incompatibility problem in international visual communi- 
cations, it is necessary to introduce some standardized 
video coding formats. Typical video formats for such 
purposes are H.261 and H.263, recommendations from 
the International Telecommunications Union-Telecom- 
munications Standards Sector (ITU-T). They provide 
two common video formats called "Common Intermedi- 
ate Format (GIF)" and "Quarter Common Intermediate 
Format (QCIF)." Some of their key specifications are 
shown in FIG. 19 to make a comparison in terms of the 
number of picture elements per line, the numt>er of lines 
per frame, and frame frequency. In this table of FIG. 19. 
the symbol "Y" represents luminance signals, and "Cr" 
and "Cb" denote color difference signals. 
[00041 FIG. 20 shows a sinrplif ied system structure 
for international digital video communications between 
two countries using different television system stand- 
ards. This video communications system interconnects 
an NTSC system 2 and a PAL system 4 by using CIF as 
a vehicle for relaying digital video signals via a commu- 
nications satellite 3. To convert the video format, a video 
signal processor 200a is disposed at the NTSC side, 
and another video signal processor 200b at the PAL 
side. The former processor 200a comprises a format 
converter 201a and a codec (coder/decoder) 202a. 
while the latter processor 200b comprises a format con- 
verter 201b and a codec 202b. Such video signal proc- 
essors 200a and 200b are implemented as part of 
videoconferencing stations or videophone terminals, for 
example. 



[0005] Consider that the NTSC system 2 attempts 
to send video information to the PAL system 4. The for- 
mat converter 201a in the video signal processor 200a 
converts incoming NTSC video signal into a CIF video 
5 stream, which conforms to the common format for com- 
munications. This CIF video stream is then encoded by 
the codec 202a and transmitted to the communications 
satellite 3 via a radio transmitter (not shown). The radio 
signal retransmitted by the communications satellite 3 
70 reaches a receiver (not shown) attached to the video 
signal processor 200b. The video signal, now in the 
form of an electrical signal, is supplied to the codec 
202b for video decoding, where the original CIF video 
stream is reconstructed. The format converter 201b 
75 converts this CIF video stream to a PAL video signal for 
use in the PAL system 4. It would be understood that 
video signal transmission in the opposite direction (i.e.. 
from PAL to NTSC) can be achieved in a similar fashion. 
[0006] As desaibed above, conventional visual 
20 communications systems use the CIF as a common for- 
mat to transport video information to remote sites. That 
is. the sender encodes video signals after converting 
from its local format to the CIF formal, and the receiver 
converts from the CIF format to its local format after 
25 decoding reception signals. In such international video 
communications, conventional systems resolve the dif- 
ference in frame frequencies by subsampling or dupli- 
cating video frames. More specifically, when a video 
signal having a certain frame frequency has to be con- 
30 verted to a lower frequency (e.g., 30 Hz to 25 Hz), the 
source frames are subsanpled. or decimated, at prede- 
termined intervals. On the other hand, when a video sig- 
nal has to be converted to a higher frequency, some 
frames are sent twice to adjust the frame frequency. 
35 [0007] However, the above-described video conver- 
sion techniques introduce some unnatural effects into 
objects' motion in a video, because of the discontinuity 
of frame pictures. Viewers may perceive the drop or 
duplication of frames as awkward motion of objects, 
40 particularly when the video contains rapid motion such 
as a soccer ball flying in a parat>ola. 

SUMMAFTt^ OF THE INVENTION 

45 [0008] Taking the above into consideration, an 
object of the present invention is to provide an appara- 
tus for converting between different video formats to 
enable visual communications between users of incom- 
patible television systems, which maintains the smooth- 
so ness of object motions by preventing unnatural effects 
from being introduced in the process of video format 
conversion. 

[0009] Further, another object of the present inven- 
tion is to provide a video format conversion method for 
55 visual communications between users of inconpatible 
television systems, which maintains the smoothness of 
object motions by preventing unnatural effects from 
being introduced in the process of video format conver- 
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sion. 

[0010] To accomplish the first object, according to 
the present Invention, there is provided an apparatus for 
converting from a first video signal In a first video format 
to a second video signal In a second video format that is s 
incompatible with the first video fbrnlat This apparatus 
comprises: a frame interpolator which produces interpo- 
lated frames from the first video signal by using motion 
vectors obtained therefrom: and a video signal genera- 
tor which produces the second video signal from the io 
interpolated frames. 

[0011] To accomplish the second object, according 
to the present Invention, there is provided a method of 
converting from a first vbeo signal in a first video format 
to a second video signal in a second video format that is is 
inconpatible with the first video format. This method 
comprises the steps of: producing interpolated frames 
from the first video signal by using motion vectors 
obtained therefrom; and producing the second video 
signal from the interpolated frames. 20 
[0012] The above and other objects, features and 
advantages of the present invention will become appar- 
ent from the following description when taken in con- 
junction with the accompanying drawings which 
illustrate preferred emkxxJiments of the present inven- 25 
tion by way of example. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0013] 30 

FIG. 1 is a conceptual view of a video format con- 
verter according to the present invention; 
FIG. 2 is a block diagram of a video codec; 
FIG. 3 is a block diagram of a frame interpolator 35 
according to a first embodiment of the present 
invention; 

FIG. 4 is a diagram showing the concept of forward- 
reference and backward-reference motion vectors; 
FIG. 5 Is a block diagram of a video signal genera- 40 
tor of the first embodiment; 

FIG. 6 Is a diagram which shows Inputs and outputs 
of frame memories; 

FIGS; 7 and 8 are diagrams showing frame interpo- 
lation processes in the first embodiment; 45 
FIG. 9 is a diagram showing the relationships 
between two video streams in the first embodiment; 
FIG. 10 is a block diagram of a frame interpolator 
according to a second embodiment of the present 
invention; so 
FIG. 11 is a block diagram of a video signal gener- 
ator of the second embodiment; 
FIG. 12 is a diagram which showvs inputs and out- 
puts of frame memories; 

FIG. 13 is a diagram showing such a situation ss 
where the proposed converter encounters a scene 
change when producing interpolated frames in nor- 
mal mode; 



FIGS; 14 and 15 are diagrams showing frame inter- 
polation processes in the second embodiment; 
FIG. 16 is a diagram showing the relationships 
between two video streams in the second embodi- 
ment; 

FIG. 17 is a flowchart showing a process of video 
format conversion according to the present inven- 
tion; 

FIG. 18 is a table showing some key specifications 
of NTSC. PAL, and SECAM; 
FIG. 19 is a table showing some key specifications 
of GIF and QCIF; and 

FIG. 20 is a diagram showing a simplified system 
structure for irrternational digital video communica- 
tions between two countries using different televi- 
sion system standards. 

DESCRIPTION OF THE PREFERRED EMBODI- 
MENTS 

[0014] Preferred embodiments of the present inven- 
tion will be descrS^ed below with reference to the 
accompanying drawings. 

[0015] FIG. 1 shows the concept of a video format 
converter according to the present invention. This pro- 
posed video format converter 1. comprising a frame 
interpolator 10 and a video signal generator 20. per- 
forms video format conversion to send visual informa- 
tion to a remote end that uses a different television 
system. Here, the term "video format conversion" refers 
to a process to convert frame frequencies. 
[001 6] According to the present invention, the frame 
interpolator 10 produces Interpolated frames from a first 
video signal given in a first video format by using 
motion vectors obtained therefrom. (Technical details of 
frame interpolation will be described later.) From the 
interpolated frames, the video signal generator 20 pro- 
duces a second video signal in a second video format 
that is different from the first video format. 
[0017] Recall that conventional video format con- 
verters use simple frame subsampling or frame inser- 
tion techniques to convert from one frame frequency to 
the other. In contrast to this, the proposed video format 
converter 1 uses frame interpolation techniques to 
obtain a new frame frequency. That is. the video format 
converter 1 is designed to generate interpolated frames 
from the first video signal, and with these interpolated 
frames, produce the second video signal having a differ- 
ent frame frequency. It should be noted here that the 
proposed converter 1 produces interpolated frames in 
an adaptive manner, taking into consideration the differ- 
ence between consecutive frames, which is repre- 
sented by motion vectors. Because of the use of 
interpolated frames to recreate frames of the second 
video signal, the video format converter 1 of the present 
invention provides smooth motion in a scene, without 
awkward frame drops or other unnatural effects. 
[0018] A more specific implementation of the 
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above-described video format converter 1 will be 
desaibed below. FIG. 2 shows a structure of a video 
codec 100 which processes video signals by using 
internationally standardized video processing algo- 
rithms. This video codec 1 00 accepts video signals from 
two cameras. camera-A and camera-B. both having a 
frame frequency fi . while its internal common video tor- 
mat has another frame frequency f2. An external video 
monitor requires incoming video signals to has/e still 
another frame frequency f3. 

[0019] A video signal switching unit no is inter- 
posed between the cameras and video signal switching 
unit 110 to select either of the two video signals sup- 
plied from the camera-A and camera-B The selected 
video signal having the lust fiame fiequency f> n> 
applied to the video codec 100 Switching fiom can>eia 
A to camera-B or vice versa, will cause an instant and 
entire transition of screen images, which is referred to 
as a "scene change " The video signal switching am* 
110 produces a scene change indication signal Ds to 
inform the video codec 100 of the occurrence of such a 
scene change. 

[0020] As the nanr>e implies, the video codec 100 
provides two major functions, video coding and decod 
ing. employing two dedicated internal data paths for 
them. Referring to its functional blocks shown in FIG 2 
one data path running from left to right serves as a 
video coder, while the other data path running from right 
to left serves as a video decoder. The following section 
will explain the elements of each data path, as well as 
their operation 

[0021] The video coding path begins with a video 
format converter i -1 . v^^hich comprises a frame interpo- 
lator 10-1 and a video signal generator 20-1 The frame 
interpolator 10-1 produces interpolated frames from a 
source video signal supplied from the video signal 
switching unit 110. by using nrxrtion vectors obtained 
from consecutve frames of the source video signal 
With the interpolated frames produced, a video signal 
generator 20-1 produces another video signal stream 
having a frame frequency f2. When a scene change 
occurs, the video signal generator 20-1 receives the 
scene change indication signal Ds from the video signal 
switching unit 110. and based on this signal, it selects 
appropriate interpolated frames that are derived from 
correlated source frames, to continuously produce the 
converted video signal. 

[0022] The video signal produced by the video for- 
mat converter 1-1 is then supplied to a coder 101a to 
perform a data compression, or reduce its data size 
More specifically, the coder 101a performs digital cosine 
transform (DCT) coding, quantization, and motion com- 
pensation, to compress spatial and temporal redundan- 
cies of video frames. After that, it applies variable-length 
coding to produce outgoing data coded in a prescribed 
format. The actual amount of the produced data may 
vary frame by frame, and hence the data rate. A trans- 
mission buffer 102a serves as temporary storage to 



smooth out such variations of the data rale. The buffer 
occupancy ratio is fed back to a coding controller 103, 
which controls the amount of coded data being pro- 
duced by the coder 101a, thus regulating the data flow 
5 to the transmission buffer 1 02a. At the final stage of the 
video coding path, a transmission coder 104a provides 
several functions necessary for data transmission. For 
instance, it inserts dummy bits when the transmission 
buffer 102a is empty, and it adds error correction code 
70 to the coded bit stream to be transmitted. 

[0023] The video decoding path, on the other hand, 
begins with a transmission decoder 104b which 
receives a coded bit stream from a remote end. It 
removes dummy bits from the received bit stream, as 
1 5 well as perfornfTS en-or check and correction. A reception 
buffer 102b regulates the flow of incoming data to 
ensure that each frame be decoded within a prescribed 
period The received bit stream is then supplied to a 
decoder 101b. where a variable-length decoding algo- 
fiihrr IS applied so as to extract each piece of coded 
data from the bit stream. The decoder 101b further exe- 
cutes dequantization, inverse-DCT, and motion-com- 
p«nsated image reconstruction, thereby expanding the 
cnrt*n data 

- [0O24] The video signal reconstructed by the 
decoder 101b has a frame frequency f2, which is not 
oorrxjattrfe with the external video monitor. For fre- 
Quercv conversion, the video signal is supplied to a 
vid€o tormat converter 1-2, which comprises a frame 

V nter poiator 10-2 and a video signal generator 20-2. The 
tiame interpolator 10-2. produces interpolated frames 
ficm tne supplied video signal by using motion vectors. 
Ftom these interpolated frames, the video signal gener- 
ator 20 2 produces a new sequence of frames, or a 

3^ video signal with a different frame frequency. f3. which 

15 compatible with the video monitor. 
1002S1 The format of incoming coded bit stream 
may Change from GIF to QCIF. or QCIF to GIF. during 
operation When such a common format switchover is 

4. erx:ountered. the video signal generator 20-2 selects 
appropriate interpolated frames that are derived from 
correlated source frames, to continue to produce the 
votjo signal with the frame frequency f3. 
[0026] A first embodiment of the present invention 
4f wflfl fw* be described below with reference to FIGS. 3 to 
9 In tr»s first embodiment, a video format converter 
reouccs the frame frequency. For illustrative purposes, it 

16 assumed here that the proposed converter is 
dew^ned to make a frame frequency conversion from 

V 30 to 25 Hz. 
[0027] Referring to a block diagram of FIG. 3, a 
fiarrw* interpolator 10a designed for video format con- 
version to a lower frame frequency is provided. This 
frame interpolator 10a produces three kinds of intsrpo- 

55 lated frames FLa. FLi^, and FLc from a first video signal. 
It employs four frame memories Ml to M4. which are 
conneaed in series to store four past frames of the first 
video signal being supplied from an external source. As 
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FIG. 3 shows, the symbol for the present incoming 
frame is "Ml in," while those for the outputs of the frame 
memories M1, M2, M3. and M4 are "Maout," "MSout," 
"M4out.- and "MSout," respectively. 
[0028] A motion detector MC produces fonward-ref- 
erence motion vectors by making a comparison 
between the frames M2out and MSout. It also produces 
backward-reference motion vectors by making a com- 
parison between the frames M1in and M2out. FIG. 4 
explains the concept of forward-reference and back- 
ward-reference motion vectors. Suppose here that the 
frame M2out is now being subjected to the motion 
detection process (i.e.. M2out is the current picture). 
When observed from this frame M2out. the frame MSout 
is regarded as its previous frame, and the frame Ml in as 
its future frame. A forward-reference nx)lion vector Vf 
represents the direction and distance of a nx>ving object 
observed between the frames M3out and M2oul On the 
other hand, a backward-reference motion vector Va rep- 
resents those observed between the framcc M2out and 
Mlin. 

[0029] Referring again to FIG. 3, the f ^anie interpo- 
lator 10a has two delay units D1 and C2 eacn d wnich 
provides a one-frame delay to given for^^rn #*>w»f«v:<» 
and backward-reference motion veaof s vt ann va That 
is. forward-reference and backward reference met on 
vectors Vf and Va appearing at the ouixii o« tt>* deiar 
unit D1 have a delay of one frame mte* w^ai «^rft^ »e«4^€Ci 
to those at the output of the motion detectoi MC wn.** 
those appearing at the output of the Oe<a> unrt c r 
a delay of two frame intervals. 

[0030] The above-noted motion vector outx/ts cf 
the motion detector MC and delay units Di and af^ 
supplied respectively to three multipliers ' la and 
11c for multiplication of vector weighting cc««c*ents C t 
C2, and 03. Here, the vector weighting coe«c»«nts Ci 
to C3 denote the mixture ratios d lonnfard 'eVtence 
motion vectors and backward-reference moton sectors 
By multiplying such coefficients, eacn muTc»'*€^ na to 
lie calculates a weighted mixture of kun^atxi t eie«enc« 
motion vectors Vf and backward-reference motion /ec 
tors Va thereby outputting weighted motior vectors MVi 
to MVS. According to those weigN«J niofion ^eclur*k 
MVI to MVS, variable delay units VDl to VD3 aopi/ 
motion compensation to the frames M3out to MOout, 
respectively. They output the resultant frames FLi to 
FL3 in phase with the three frames M3out to MSout. 
respectively 

[0031] Switches SW1 to SW3 select video frames 
according to a motion/still selection command provided 
from an external source. More specifically, they select 
the outputs of the variable delay units VDl to VD3 when 
the motion/still selection command indicates motion pic- 
ture mode. In turn, they select the outputs of the frame 
memories M2 to M4 when the command indicates still 
picture mode. Picture elements of the selected frames 
are then supplied to multipliers 12a to 12c for multiplica- 
tion of frame weighting coefficients kl to k3. Here, the 



coefficients kl to k3 give a mixture ratio of the three 
selected frames. The resultant weighted frame signals 
FL1-m to FL3-m are supplied to adders at the next 
stage for frame summation. 

5 [0032] There are three adders 1 3a to 1 3c to obtain 
three interpolated frames. The first adder 13a produces 
an interpolated frame FLa by adding the last two 
weighted frames FL2-m and FL3-m. The second adder 
13b produces another interpolated frame FLb by adding 

10 all the three weighted frames FLI -m. FL2-m and FL3-m. 
The third adder I3c produces still another interpolated 
frame FLc by adding the first two weighted frames FLI - 
m and FL2-m. 

[0033] The next section wilt now explain a specific 

15 Structure of a video signal generator designed for video 
format conversion to a lower frame frequency (e.g., 30 
Hz to 25 Hz). FIG. 5 is a block diagram of a vkJeo signal 
generator 20a. This video signal generator 20a com- 
prises a switch SW21a, a FIFO 22a, a FIFO controller 

20 24a, and a frame switching controller 27. 

[0034] The frame switching controller 27 comprises 
difference detectors 27a to 27d and a switching control- 
ler 27e. The difference detector 27a detects differences 
between the frames Mlin and M2out. The difference 

?£ detector 27b detects differences between the frames 
M2out arxj M3out. The difference detector 27c detects 
differences between the frames M3out and M4out. The 
difference detector 27d detects differences between the 
frames M4out and MSout Examining the detected dif- 

> ferences. the switching controller 27e finds the maxi- 
mum difference value. This Information permits the 
Stfvltching controller 27e to determine at which frame 
position the scene change or common format switcho- 
ver has occurred, based on a scene change indication 
Signal Ds or a common format switching command Df 
supplied from external sources. The switching controller 
27e thus outputs an appropriate switching control com- 
mand SWC to the switch SW21a. 
[0035] The switch SW2la has three input terminals 

A "a," "b." and "c" to receive Interpolated frames FLa to 
FLC. the outputs of the adders 13a to 13c described In 
FIG. 3. The switch SW21a selects either one of the 
thiee inputs according to the switching control com- 
mand SWC as will be described later. 

4f [0036] The FIFO 22a serves as a buffer memory for 
the interpolated frames being supplied from the switch 
SW21a at the rate of 30 Hz. It controls their output tim- 
ings under the control of the FIFO controller 24a. so that 
interpolated frames will be sent out at 25 Hz. 

5.' [0037] The switch SW21a is controlled as follows. 
The switch SW2la has two modes of operation: normal 
mode and transitional mode. In normal mode, the switch 
SW21a keeps its contact sw-la at the second position 
t)." and thus the FIFO 22a receives interpolated frames 

55 FLb at a rate of 30 Hz. The FIFO 22a decimates some 
of the received frames at predetermined intervals, 
thereby converting the frame frequency down to 25 Hz. 
[0038] Transitional mode, as opposed to the normal 
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mode, is selected at scene changes, as well as in the 
case that the common video format is switched from 
GIF to QCIF or vise versa (i.e., common fonnat switcho- 
ver). While later explanation assumes the occurrence of 
a scene change, those skilled in the art will appreciate - 
that the same will apply to the operation In the case of 
common format switchover. 

[0039] FIG. 6 shows inputs and outputs of the frame 
memories M1 to M4 (FIG. 3). The symbol "Mlin" is used 
to represent a source frame being entered to the first 
frame memory Ml, while "M2out" to "MSout" represent 
the output pictures of the frame memories Ml to M4, 
respectively. Those five consecutive pictures represent 
a part of a given frame sequence. FIG. 6 shows three 
instances of such frame snapshots taken at times tO, t1 . 
and t2, and in this progression, the pictures are shifted 
by one frame toward the right-hand side of FIG. 6. 
[0040] This example video sequence of FIG. 6 
includes a scene change, which actually happened at 
some time before tO. The switching controller 27e is 
designed to initiate transitional mode operation when 
such a scene change is observed between the frames 
M3out and M4out. Here, the switching control command 
SWC serves as a message to inform the switch SW21 a 
that a scene change has reached that critical point. This 
message makes the switch SVV21a enter the transi- 
tional mode and handle the scene change at times tO 
and t1. That is. the transitional mode operation lasts 
only two frame intervals, and the switch SW21a returns 
to normal mode at time t2 to resume normal switching 
operations. 

[0041] At time tO (i.e.. the first half of transitional 
mode operation), the switch SW21a first has to confine 
the source of interpolated frames to frames captured 
before the occun-ence of the scene change. To this end, 
the switch SW21a moves its contact sw-la to the first 
position "a" according to the switching control command 
SWC, thereby selecting an interpolated frame FLa 
derived from the frames M4out and M5out (i.e., old 
scene). After that, at time t1. the switch SW21a moves 
the contact sw-1 a to the third position "c" so as to select 
interpolated frames derived only from a new scene. 
Accordingly an interpolated frame FLc produced from 
the frames M3out and M4out is selected. At time t2, the 
switch SW21a resumes its normal mode operation by 
returning its contact sw-la to the second position "b." 
again selecting interpolated frames FLb being produced 
from M3out, M4out. and MSout. 

[0042] The switch SW21a is designed to change 
the position of its contact sw-la step by step (i.e.. "b"- 
"c''-"a"-"b") according to each switching control com- 
mand SWC. However, it is not intended that the inven- 
tion be limited to this structure, but the swvitch SW21a 
can also be configured to do the same autonomously at 
predetermined intervals, once it receives the first 
switching control command SWC. 
[0043] The next section will now describe a process 
of producing interpolated frames for video format con- 



version to a lower frame frequency (e.g.. 30 Hz to 25 
Hz). 

[0044] FIGS. 7 and 8 present a frame interpolation 
process including no scene change or common format 

s switchover. The symbol "Fin" represents a source frame 
sequence having a frame frequency of 30 Hz that is 
given to the frame interpolator 1 0a. For the sake of con- 
venience, source frames are labeled "Fl" to T6" in a 
cyclic manner. Consider that the first instance of "Fl" 

10 appears at the output of the fourth frame memory M4. 
Then the frames F2, F3, and F4 are at the outputs of the 
frame memories M3, M2, and Ml . respectively and F5 
at the input of the first frame memory Ml. At the next 
cycle, the frame F2 appears at the output of the frame 

15 memory M4, since the frame memory contents are 
shifted forward by one frame. Likewise, the frames F3, 
F4, and F5 are now at the outputs of the frame memo- 
ries M3. M2, and Ml . respectively, and F6 at the Input of 
the first frame memory Ml . 

20 [0045] On the other hand, the symbol "Fout" shows 
a series of interpolated frames FLb produced by the 
video signal generator 20a when the switch SW21 a is in 
norma! mode (i.e., its contact is at the second position 
"b"). The interpolated frames Fout are labeled "F1a" to 

25 "F5a" in a cyclic manner 

[0046] In the process shown in FIGS. 7 and 8. inter- 
polated frames are produced from the following combi- 
nations of source frames: 

30 • Fl a produced from Fl , F2. and F3 
F2a produced from F2, F3. and F4 
F3a produced from F3. F4. and F5 
F4a produced from F5. F6. and Fl 
• F5a produced from F6, Fl , and F2 

35 

Note here that, although the frame interpolator 10a pro- 
duces interpolated frames from frames F4 to F6. the 
FIFO 22a in the video signal generator 20a will never 
output them. That is, the proposed video format con- 
40 verter subsamples the produced interpolated frames to 
reduce its frame frequency. 

[0047] FIG. 9 depicts the above-noted relationships 
between source frames (30 Hz) and sul^sanpled 
frames (25 Hz). Each bold solid line represents a frame. 

45 while broken lines indicate unit time intervals. The inter- 
polated frames Fout shown in FIGS. 7 and 8 are fed to 
the FIFO 22a at the rate of 30 Hz. The FIFO 22a regu- 
lates the intervals of output frames so that they will be 
sent out at 25 Hz. 

50 [0048] A second embodiment of the present inven- 
tion will now be described below, w'rth reference to 
FIGS. 10 to 17. As opposed to the first embodiment, a 
video format converter proposed of the second embod- 
iment is designed to raise the frame frequency. For iilus- 

55 trative purposes, it is assumed here that the proposed 
converter makes a frame frequency conversion from 25 
Hz to 30 Hz. 

[0049] Referring first to a block diagram of FIG. 10, 
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a frame interpolator 1 0b designed to raise the frame fre- 
quency will be described below. This frame interpolator 
10b produces four kinds of interpolated frames FLa. 
FLb, FLc, and FLxl from a first video signal. This frame 
interpolator 10b differs from the frame interpolator 10a 
of FIG. 3 in that a switch SW4 and an adder 13d are 
newly employed. Since the other functional elements of 
the frame interpolator 10b are common to those of the 
frame interpolator I0a. the following explanation will 
focus on their differences, while maintaining like refer- 
ence numerals to like elements. 
[0050] The adder I3d produces a fourth interpo- 
lated frame FLd by adding the weighted frames FL1 -m 
and FL2-m. The switch SW4 is controlled through a 
switching control command SWCa. While being nor- 
mally closed, this switch SW4 is turned off when a 
scene change or common format switchover is 
detected, in which case the interpolated frame FLxi is 
identical to the frame FLI-m, since the adder 13d 
becomes transparent. More details of the switch SW4 
and switching control command SWCa will be provided 
later. 

[0051 ] Referring next to FIG. 1 1 , a video signal gen- 
erator 20b designed to raise the frame frequency will be 
described below. This video signal generator 20b com- 
prises two switches SW21b and SW22b. two FIFOs 22b 
and 25b. a FIFO controller 24b, a first frame switching 
controller 27-1 . and a second frame switching controller 
26b. 

[0052] The first frame switching controller 27-1 has 
difference detectors 27a to 27d and a switching control- 
ler 27e-1. The difference detector 27a detects differ- 
ences between the frames Ml in and M2out. The 
difference detector 27b detects differences between the 
frames M2out and M3out. The difference detector 27c 
detects differences between the frames M3out and 
M4out. The difference detector 27d detects differences 
between the frames M4out and MSout. By examining 
the detected differences, the switching controller 27e-l 
finds the maximum difference value. This information 
permits the switching controller 27e-1 to determine at 
which frame position the scene change or common for- 
mat switchover has occurred, based on a scene change 
indication signal Ds or a common format switching com- 
mand Df supplied from external sources. The switching 
controller 27e-1 thus outputs an appropriate switching 
control command SWC to control the first switch 
SW21b. It further produces another switching control 
command SWCa to control the switch SW4 in the frame 
interpolator 10b. 

[0053] The first switch SW21b has three Input ter- 
minals "a." "b," and "c" to receive three kinds of interpo- 
lated frames FLa. Fl_b, and FLx:. the outputs of the 
adders 13a to 13c shown in FIG. 10. This first switch 
SW21b selects either one of the three inputs according 
to the switching control command SWC, and sends the 
selected frames to the first FIFO 22b. On the other 
hand, the second FIFO 25b receives another kind of 



Interpolated frames, FLd, from the frame interpolator 
10b. Note that those Interpolated frames are generated 
at 30 Hz. i.e., the rate of source video frames. The FIFO 
controller 24b supplies control signals to the two FIFOs 

5 22b and 25b to regulate the flow of outgoing frames, so 
that they will be output at 25 Hz. 
[0054] The outputs of the two FIFOs 22b and 25b 
are then supplied to two input terminals "d" and "e" of 
the second switch SW22b, respectively. According to 

10 frame switching commands Dc from the controller 26b. 
the second switch SW21b chooses one of the two input 
signals as final output of the video signal generator 20b. 
[0055] Referring to FIGS. 10 to 12, the following 
section will describe how the switches SW4, SW21b, 

rs and SW22b operate in the second embodiment. As in 
the first embodiment, those switches have two modes of 
operation: normal mode arKi transitional mode. The lat- 
ter mode is selected In the case of scene changes or 
common format switchover. Although the case of a 

20 scene change will be discussed later, those skilled in 
the art will appreciate that the same will apply to com- 
mon format switchos/er. 

[0056] FIG. 12 shows inputs and outputs of the 
frame memories Ml to M4 (FIG. 10). The symbol Ml in 

25 is used to represent a source frame being entered to the 
first frame memory M1 . while M2out to M5out represent 
the output pictures of the frame memories Ml to M4. 
respectively. These five consecutive pictures represent 
a part of a given frame sequence. FIG. 12 shows four 

30 instances of such frame snapshots taken at times tO. t1 , 
t2. and t3. and in this progression, the pictures are 
shifted by one frame, toward the right-hand side of FIG. 
12. 

[0057] In normal mode, the SW4 is closed and the 
35 first switch SW21b keeps its contact sw-1b at the sec- 
ond position l)." On the other hand, the second switch 
SW22b connects its contact sw-2b at the first position 
"d" according to the frame switching command Dc, 
which allows the produced frames to be transmitted 
40 through the First Fl FO 22b and second switch SW21 b in 
the following sequence: 

FLb-1 produced from frames M3out to MSout at 
time to 

45 • FLb-2 produced from frames M3out to MSout at 
time t1 

FLb-3 produced from frames MSout to MSout at 
time t2 

FLb-4 produced from frames MSout to MSout at 
50 time tS 

[0058] When the Interpolated frame FLb-3 is out- 
put, the adder 13d produces another Interpolated frame 
FLd from the frames MSout and M4out and supplies it to 
55 tiie second FIFO 25b. After the interpolated frame FLb- 
4 is sent out. the frame switching controller 26b alters its 
command output Dc to make the first FIFO 22b move its 
contact sw-2b to the position *'e" so as to select the 
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frame FLd hekJ in the second FIFO 25b. The frame 
switching controller 26b then commands the second 
switch SW21 b to return its contact sw-2b to the position 
"d." thus repeating the above-described operation. 
[0059] If a scene change is encountered, the pro- s 
posed video format converter enters to transitional 
mode. The operation in this mode will be described 
below with reference to FIGS. 6, 10, and 1 1 . 
[0060] As described earlier, the example video 
sequence of FIG. 6 includes a scene change. The io 
switching controller 27e-1 (FIG. 11) is designed to initi- 
ate transitional mode operation, when such a scene 
change Is observed between the frames M3out and 
M4out. Here, the switching control command SWC 
serves as a message to inform the first switch S>/V2lb is 
that the scene change has reached thai cntfcal point. 
This message makes the first switch SW2ib enter the 
transitional mode and handle the scene change at tmes 
tOandtl. The first switch SW21bthen returns to normal 
mode at time t2 to resume normal switching ooerations 
[0061 ] At time tO, the first switch SW2 1 0 has to con- 
fine the source of interpolated frames to frames cap- 
tured before the occurrence of the scene change To 
this end. the first switch SW21b moves its oonuict a* - ib 
to the first position "a" according to the switcfitng control 
command SWC. thereby selecting an inter poiated 
frame FLa produced from the frames M4oct ana MSout 
At the same time, the second switch sets its 

contact sw-2b to the first position "d" accoftJng to tt>e 
frame switching command Dc. As a resutt t^e inte^po x 
lated frame FLa is sent out. 

[0062] At time t1 the first switch SW?it) *«s. to 
select an interpolated frame produced onty from tr»e 
new scene. The first switch SW2ib thus nxves con 
tact sw-lb to the third position "c." thereby seecring an 
interpolated frame FLc produced from the tiames M jout 
and M4out. The second switch SW22b maniatfis its 
contact position "d" according to the frame switctmg 
command Dc. thus serKfing out the selected rnierpo 
lated frame FLc. < 
[0063] At time t2. the first switch SW2 1 b returns its 
contact sw-1 b to the second position V to resume nof- 
mal PTKxje operation, again selecting interpolated 
frames FLb. The second switch SW22b keeps rts con- 
tact position "d" according to the frame switching com 4t 
mand Dc. thus sending out the seleaed interpolated 
frame FLb. 

[0064] As discussed above, the first switch S\A/2ib 
is designed to change the position of its contact sw-ib 
step by step (i.e., 'b''-"c''-"a*'-"b") according to each 
switching control command SWC However, it is not 
intended that the invention be limited to this structure, 
but the first switch SW21b can also be configured to do 
the same autonomously at predetermined intervals, 
once it receives the first switching control command 55 
SWC. 

[0065] FIG. 13 shows such a situation where the 
proposed converter encounters a scene change when 



producing an interpolated frame FLd in normal mode. 
Although it is normally closed, the switch SW4 is 
designed to become open when a scene change is 
observed at a particular point shown in FIG. 13. This 
means that the aforementioned switching control com- 
mand SWCa serves as a message to inform the switch 
SW4 that a scene change has reached that critical 
point. With the switch SW4 turned off, the adder 13d 
produces an Interpolated frame FLd solely from the 
frame M3out. 

[0066] Referring now to FIGS. 14 and 15. the next 
section will describe a process of producing interpo- 
lated frames to convert the video format to a higher 
frame frequency (e.g., 25 Hz to 30 Hz). This illustrated 
process, however, includes no scene change or com- 
mon format switchover. 

[0067] The symbol "Fin" represents a source frame 
sequence having a frame frequency of 25 Hz that is 
given to the frame interpolator 1 0b. For the sake of con- 
venience, source frames are labeled "Fl" to ''F5'' in a 
cyclic manner. Consider that the first instance of "F1" 
appears at the output of the fourth frame memory M4. 
Then the frames F2. F3. and F4 are at the outputs of the 
frame memories M3, M2, and Ml . respectively, and F5 
at the input of the first frame menrrory Ml . 
[0068] At the next cycle, the frame F2 appears at 
the output of the frame memory M4. since the frame 
memory contents are shifted forward by one frame. 
Likewise, other frames F3. F4. and F5 are now at the 
outputs of the frame memories M3, M2, and Ml, 
respectively, and the second instance of F1 appears at 
the input of the first frame memory Ml . 
[0069] On the other hand, the symbol "Foutl" 
shows a series of interpolated frames FLb produced by 
t^e video signal generator 20b when the first switch 
SW2lb is in normal mode (i.e., its contact is at the sec- 
ond position "b"). Those frames Fouti are labeled 
T- 1 a." ■'F2a.'* "F3a." "FSa." and "F6a" in a cyclic manner, 
f-jrther. the symbol "Fout2" shows additional interpo- 
lated frames FLd produced by the adder 13d in the 
frame interpolator 10b. In FIGS. 14 and 15, such inter- 
polated frames are labeled '*F4a." 
[0070] The above process as a whole generates six 
interpolated frames in the following sequence during 
f rve source frame intervals: 

Fla produced from F1, F2, and F3 
F2a produced from F2, F3, and F4 
F3a produced from F3, F4, and F5 
F4a produced from F4 and F5 
• F5a produced from F4, F5, and F1 
F6a produced from F5. F1 . and F2 

Note here that the frames F3a and F4a are produced at 
the same time. The flow of produced frames is regu- 
lated by controlling the FIFOs, thus obtaining final out- 
put at a frame frequency of 30 Hz. 
[0071] FIG. 16 depicts the relationships between 
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source frame sequence (25 Hz) and converted frame 
sequence (30 Hz). Each bold solid line represents a 
frame, while broken lines indicate unit time intervals. 
The interpolated frames Fouti and Fout2 shown in 
FIGS. 14 and 15 are fed to the FIFOs 22b and 25b for 
output flow control. The FIFOs 22b and 25b serve as 
buffer storage to regulate the intervals of output frames 
so that they will be sent out at 30 Hz. 
[0072] As described above, the video format con-, 
verter 1 of the present invention uses frame interpola- 
tion techniques to obtain a new frame frequency. That 
is. the video format converter 1 is designed to adaptively 
generate interpolated frames from the first video signal 
by considering motion vectors obtained therefrom. A 
second video signal is produced from the interpolated 
frames, so that it will have a different frame frequency. 
Accordingly, the video format converter 1 of the present 
invention provides smooth motion in a scene, eliminat- 
ing discontinuity in a frame sequence, which could be 
introduced in the process of frame frequency conver- 
sion. 

[0073] The following section will now describe a 
video format conversion method according to the 
present invention. FIG. 17 is a flowchart showing a 
process to convert from a first video signal in a first 
video format to a second video signal in a second video 
format that is inconrpatible with the first video format. 
The process comprises the following steps: 

(51) Interpolated frames are produced from the first 
video signal by using motion vectors of the first 
video signal. 

(52) The second video signal is produced from the 
interpolated frames. 

[0074] More specifically, the difference between the 
two video formats lies in their frame frequencies. That 
is. the proposed method performs frame interpolation of 
the first video signal having a first frame rate, thereby 
generating the second video signal having a second 
frame frequency. 

[0075] The above-described embodiments of the 
present invention is now summarized as follows. 
According to the present invention, a method and appa- 
ratus for converting between different video formats are 
proposed. They produce interpolated frames from a first 
video signal, and from these interpolated frames, gener- 
ate a second video signal that is incompatible with the 
first video signal. With appropriate frame interpolation 
techniques, the present invention promises smooth 
video motion in visual communications between remote 
users of different television systems, eliminating unnat- 
ural effects which could be introduced in the process of 
video format conversion. 

[0076] The foregoing is considered as Illustrative 
only of the principles of the present invention. Further, 
since numerous modifications and changes will readily 
occur to those skilled in the art. it is not desired to limit 



the invention to the exact construction and applications 
shown and described, and accordingly all suitable mod- 
ifications and equivalents may be regarded as falling 
within the scope of the invention in the appended claims 
5 and their equivalents. 

Claims 

1 . An apparatus for converting from a first video signal 
10 in a first video format to a second video signal in a 

second video format that is incompatible with, the 
first video format, comprising: 

frame interpolating means for producing inter- 
75 polated frames from the first video signal by 

using motion vectors obtained therefrom; and 
video signal generating means for producing 
the second video signal from the interpolated 
frames. 

20 

2. The apparatus according to claim 1 . wherein said 
frame interpolating means produces a plurality of 
frames by using the motion vectors for motion com- 
pensation, multiplies the produced frames by 

25 weighting coefficients, obtains weighted sums by 
adding the weighted frames, and outputs the 
weighted sums as the interpolated frames. 

3. The apparatus according to claim 1 . wherein said 
30 frame interpolating means uses forward-reference 

motion vectors and backward-reference motion 
vectors as the motion vectors. 

4. The apparatus according to claim 1 . wherein ssiid 
35 frame interpolating means produces the interpo- 
lated frames, while switching between still picture 
mode and motion picture mode. 

5. The apparatus according to claim 1 , wherein: 

40 

the first video format has a first frame fre- 
quency; 

the second video format has a second frame 
frequency that is different from the first frame 
45 frequency; and 

said video generating means produces the 
second video signal having the second frame 
frequency. 

so 6. The apparatus according to claim 1 . wherein: 

said video generating means detects differ- 
ences between consecutive frames of the first 
video signal; and 
55 when a scene change is indicated, said video 

generating means produces the second video 
signal by choosing, based on the detected dif- 
ferences, such interpolated frames that are 
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derived from either group of correlated source 
frames, those before the scene change or 
those after the scene change, in the first video 
signal. 

5 

7. The apparatus according to claim 1, wherein: 

said video generating means detects differ- 
ences between consecutive frames of the first 
video signal; and io 
when common format switchover is indicated, 
said video generating means produces the 
second video signal by choosing, based on the 
detected differences, such interpolated frames 
that are derived from either group of correlated is 
source frames, those before the common for- 
mat switchover or those after the common for- 
mat swvitchover. in the first video signal. 

8. A method of converting from a first video signal in a 20 
first video format to a second video signal in a sec- 
ond video format that is incompatible with the first 
video format, comprising the steps of: 

(a) producing interpolated frames from the first 25 
video signal by using motion vectors obtained 
therefrom; and 

(b) producing the second video signal from the 
interpolated frames. 

30 

9. The method according to claim 8, wherein: 

the first video format has a first frame fre- 
quency; and 

the second video format has a second frame 35 
frequency that is different from the first frame 
frequency. 
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